Methods and devices for determining visual fatigue of three-dimensional image or video and computer readable storage medium

ABSTRACT

The embodiments of the present disclosure propose a method and device for determining a visual fatigue of a three dimensional (3D) image or a 3D video and a computer readable storage medium. The method comprises: determining depth-of-field values of at least a part of pixels of the 3D image or at least one frame of 3D image in the 3D video; and determining the visual fatigue of the 3D image or the 3D video according to the depth-of-field values.

CROSS-REFERENCE TO RELATED APPLICATION(S)

The present application claims priority to the Chinese PatentApplication No. 201710066363.9, filed on Feb. 6, 2017, entitled “METHODSAND DEVICES FOR DETERMINING VISUAL FATIGUE OF THREE-DIMENSIONAL IMAGE ORVIDEO,” which is incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present disclosure generally relates to the field of display, andmore particularly, to a method and device for determining visual fatiguefor a three-dimensional (3D) image or video and a computer readablestorage medium.

BACKGROUND

With the progress of technology, movie theaters are no longer the onlyplaces to watch 3D movies. In fact, it is also possible to watch a 3Dvideo using a television (in cooperation with stereo glasses) at home,or watch a 3D video with a mobile phone (for example, in cooperationwith a Head-Mounted Device (HMD)).

However, a common phenomenon is that whenever a 3D video is watched fora period of time, people may usually feel discomfort symptoms such asdizziness, vomiting, dry eyes etc., which is slightly similar to thesymptoms of carsickness, and is a kind of motion sickness. Morespecifically, it involves a motion sickness for which a motion isobserved by eyes but is not felt by the body, which is sometimes alsoreferred to as visual fatigue.

SUMMARY

However, there is currently no objective solution of determining avisual fatigue for a 3D image or 3D video. To this end, a method anddevice for determining a visual fatigue of a 3D image or video and acomputer-readable storage medium according to an embodiment of thepresent disclosure are proposed.

According to a first aspect of the present disclosure, there is proposeda method for determining a visual fatigue of a 3D image or a 3D video.The method comprises: determining depth-of-field values of at least apart of pixels of the 3D image or at least one frame of 3D image in the3D video; and determining the visual fatigue of the 3D image or the 3Dvideo according to the depth-of-field values.

In some embodiments, the step of determining depth-of-field values of atleast a part of pixels of the 3D image or at least one frame of 3D imagein the 3D video comprises: determining a parallax of each of the atleast a part of the pixels; and determining a depth-of-field value of acorresponding pixel according to the parallax. In some embodiments, thestep of determining the visual fatigue of the 3D image or the 3D videoaccording to the depth-of-field values comprises: determining spatialdepth-of-field differences between spatial adjacent pixels in the atleast a part of the pixels according to the depth-of-field values;determining a spatial visual fatigue of the 3D image or the 3D videoaccording to the spatial depth-of-field differences; and determining thevisual fatigue at least partly according to the spatial visual fatigue.In some embodiments, the spatial adjacent pixels comprise one or moreadjacent or spaced adjacent pixels in a spatial direction. In someembodiments, the step of determining a spatial visual fatigue of the 3Dimage or the 3D video according to the spatial depth-of-fielddifferences comprises: determining a standard deviation of all thespatial depth-of-field differences as the spatial visual fatigue of the3D image or the 3D video. In some embodiments, the step of determiningthe visual fatigue of the 3D video according to the depth-of-fieldvalues comprises: calculating temporal depth-of-field differencesbetween corresponding pixels in two frames of 3D image in the 3D video;determining a temporal visual fatigue of the 3D video according to thetemporal depth-of-field differences; and determining the visual fatigueat least partly according to the temporal visual fatigue. In someembodiments, the two frames of 3D image are two frames of 3D image whichare adjacent or spaced adjacent to each other in a forward timedirection and/or in a backward time direction. In some embodiments, thestep of determining a temporal visual fatigue of the 3D video accordingto the temporal depth-of-field differences comprises: determining astandard deviation of all the temporal depth-of-field differences as thetemporal visual fatigue of the 3D video. In some embodiments, the stepof determining the visual fatigue of the 3D image or the 3D videoaccording to the depth-of-field values comprises: calculating a standarderror between the depth-of-field values of the at least a part of thepixels and predetermined depth-of-field values as a standard visualfatigue for the at least a part of the pixels; and determining thevisual fatigue at least partly according to the standard visual fatigue.In some embodiments, the step of determining the visual fatigue of the3D image or the 3D video according to the depth-of-field valuescomprises: determining the visual fatigue according to any two or moreof a spatial visual fatigue, a temporal visual fatigue, and a standardvisual fatigue. In some embodiments, before the step of determiningdepth-of-field values of at least a part of pixels of the 3D image or atleast one frame of 3D image in the 3D video, the method furthercomprises dividing the 3D image or at least one frame of 3D image in the3D video into multiple partitions, wherein the step of determiningdepth-of-field values of at least a part of pixels of the 3D image or atleast one frame of 3D image in the 3D video and the step of determiningthe visual fatigue of the 3D image or the 3D video according to thedepth-of-field values are performed for at least one of the multiplepartitions, to determine visual fatigues of corresponding partitionsrespectively; and determining the visual fatigue of the 3D image or atleast one frame of 3D image in the 3D video according to the visualfatigues of the at least one partition. In some embodiments, wherein thestep of determining the visual fatigue of the 3D image or at least oneframe of 3D image in the 3D video according to the visual fatigues ofthe at least one partition comprises: determining a weight of acorresponding partition according to a size and/or location of each ofthe at least one partition; and determining the visual fatigue of the 3Dimage or at least one frame of 3D image in the 3D video according tovisual fatigues of various partitions and corresponding weights of thepartitions.

According to a second aspect of the present disclosure, there isproposed a device for determining a visual fatigue of a 3D image or a 3Dvideo. The method comprises: a depth-of-field value determination unitconfigured to determine depth-of-field values of at least a part ofpixels of the 3D image or at least one frame of 3D image in the 3Dvideo; and a visual fatigue determination unit configured to determinethe visual fatigue of the 3D image or the 3D video according to thedepth-of-field values.

According to a third aspect of the present disclosure, there is proposeda device for determining a visual fatigue of a 3D image or a 3D video.The method comprises: a processor, a memory having instructions storedthereon, which, when executed by the processor, cause the processor to:determine depth-of-field values of at least a part of pixels of the 3Dimage or at least one frame of 3D image in the 3D video; and determinethe visual fatigue of the 3D image or the 3D video according to thedepth-of-field values.

In some embodiments, the instructions, when executed by the processor,further cause the processor to: determine a parallax of each of the atleast a part of the pixels; and determine a depth-of-field value of acorresponding pixel according to the parallax. In some embodiments, theinstructions, when executed by the processor, further cause theprocessor to: determine spatial depth-of-field differences betweenspatial adjacent pixels in the at least a part of the pixels accordingto the depth-of-field values; determine a spatial visual fatigue of the3D image or the 3D video according to the spatial depth-of-fielddifferences; and determine the visual fatigue at least partly accordingto the spatial visual fatigue. In some embodiments, the instructions,when executed by the processor, further cause the processor to:calculate temporal depth-of-field differences between correspondingpixels in two frames of 3D image in the 3D video; determine a temporalvisual fatigue of the 3D video according to the temporal depth-of-fielddifferences; and determine the visual fatigue at least partly accordingto the temporal visual fatigue. In some embodiments, the instructions,when executed by the processor, further cause the processor to:calculate a standard error between the depth-of-field values of the atleast a part of the pixels and predetermined depth-of-field values as astandard visual fatigue for the at least a part of the pixels; anddetermine the visual fatigue at least partly according to the standardvisual fatigue. In some embodiments, the instructions, when executed bythe processor, further cause the processor to: determine the visualfatigue according to any two or more of a spatial visual fatigue, atemporal visual fatigue, and a standard visual fatigue. In someembodiments, the instructions, when executed by the processor, furthercause the processor to: determine depth-of-field values of at least apart of pixels for at least one of the multiple partitions, anddetermine visual fatigues of corresponding partitions according to thedepth-of-field values; and determine the visual fatigue of the 3D imageor at least one frame of 3D image in the 3D video according to thevisual fatigues of the at least one partition.

Further, according to fourth aspect of the present disclosure, there isproposed a non-transitory computer readable storage medium for storing acomputer program, which, when executed by a processor, causes theprocessor to perform the method according to the first aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other purposes, features and advantages of the presentdisclosure will become more apparent from the following description ofsome embodiments of the present disclosure when taken in conjunctionwith the accompanying drawings in which:

FIG. 1 is an exemplary diagram illustrating how to calculate adepth-of-field value according to a parallax according to an embodimentof the present disclosure.

FIG. 2 is a flowchart of an exemplary method for determining a visualfatigue of a 3D image or 3D video according to an embodiment of thepresent disclosure.

FIG. 3 is a diagram illustrating an exemplary partitioning schemeaccording to an embodiment of the present disclosure.

FIG. 4 is a diagram illustrating exemplary pixels used for calculating aspatial visual fatigue according to an embodiment of the presentdisclosure.

FIG. 5 is a diagram illustrating exemplary pixels used for calculating atemporal visual fatigue according to an embodiment of the presentdisclosure.

FIG. 6 is a block diagram illustrating an exemplary device fordetermining a visual fatigue of a 3D image or 3D video according to anembodiment of the present disclosure.

FIG. 7 is a diagram illustrating a hardware arrangement of the deviceshown in FIG. 6 according to an embodiment of the present disclosure.

DETAILED DESCRIPTION

Some embodiments of the present disclosure will be described in detailbelow with reference to the accompanying drawings, in which details andfunctions which are not necessary for the present disclosure are omittedin the description in order to prevent confusion in the understanding ofthe present disclosure. In the present specification, the followingdescription of various embodiments for describing the principles of thepresent disclosure is illustrative only and should not be construed aslimiting the scope of the disclosure in any way. The followingdescription of the drawings, with reference to the accompanyingdrawings, is provided to assist in a comprehensive understanding of theexample embodiments of the disclosure as defined by the claims and theirequivalents. The following description includes many specific details toassist in the understanding, but such details are to be regarded asmerely exemplary. Accordingly, those of ordinary skill in the art willrecognize that numerous changes and modifications can be made to theembodiments described herein without departing from the scope and spiritof the present disclosure. In addition, descriptions of well-knownfunctions and structures are omitted for clarity and conciseness. Inaddition, the same reference numerals are used for the same or similarfunctions and operations throughout the accompanying drawings.

Hereinafter, the present disclosure is described in detail, by taking ascene in which the present disclosure is applied to an electronic deviceas an example. However, the present disclosure is not limited thereto,and the present disclosure may also be applied to any suitable device.With respect to the electronic device, the present disclosure is notlimited to a specific operating system of the electronic device, and mayinclude, but is not limited to, iOS, Windows Phone, Symbian, Android,Windows, Linux, etc. Different electronic devices may use the sameoperating system, or may use different operating systems.

In the present disclosure, the terms “comprising” and “including” andtheir derivatives are intended to be inclusive instead of beinglimiting, and the term “or” is inclusive, which means “and/or”. In thefollowing, some of the terms to be used in the present disclosure willbe explained firstly.

Stereoscopic images or three-dimensional (3D) images: in general,current common stereoscopic (3D) images are typically implemented bygenerating two different planar images for left and right eyes which aresubstantially the same but have slight differences (i.e., “parallax”hereinafter) so as to enable a human brain to have stereoscopic feeling.Therefore, a 3D image typically comprises a left eye image for the lefteye and a right eye image for the right eye.

3D video: a series of temporally consecutive 3D images.

Parallax: A positional deviation of the same object between the left eyeimage and the right eye image in the 3D image.

Depth of field or depth: a vertical distance between a 3D object (or oneof pixels in the 3D object) observed by a user in the 3D image or 3Dvideo and a straight line between the user's eyes. In general, the depthof field here is slightly different from definition of a depth of fieldin the field of photography. The depth of field here is not a clearimaging range as in the field of photography, but is a specific value.In the field of 3D computing, this value is generally referred to as a Zvalue. In addition, it should be noted that one of pixels in the 3Dimage or 3D video mentioned here may refer to corresponding pixels inthe left eye image and the right eye image, i.e., a left eye pixel and aright eye pixel, and thus a depth of field of a certain pixel in the 3Dimage/video actually refers to a depth of field of a 3D pixel formed bythe corresponding pixels in the left eye image and the right eye image,which may be determined by, for example, the method described below,according to a parallax.

In general, it may generally be considered that a fatigue of human eyescaused by the 3D image or video mainly results from a change in aparallax in the 3D image or video. When 3D objects are observed by thehuman eyes, they eyes needs to adjust the curvature of their crystallinelens through the ciliaris, so as to clearly image different distant andnear 3D objects. Therefore, when a highly frequent change in the depthof fields occurs in the 3D image or video to be observed, it isnecessary for the human eyes to frequently adjust the crystalline lensfor clear imaging, which results in an increased fatigue of the eyes,and then leads to a corresponding fatigue of the brain. According tothis discovery, an objective method for assessing and/or determining avisual fatigue of a 3D image or video according to an embodiment of thepresent disclosure is proposed. It is to be noted, however, that thepresent disclosure is not limited thereto but may also be applied toother similar fields or applicable fields.

In some embodiments of the present disclosure, a method and a device fordetermining a visual fatigue of a 3D image or video and acomputer-readable storage medium are generally proposed, which may besummarized as follows. Firstly, a depth-of-field value of each pixel inthe 3D image is determined. Then, one or more of a temporal visualfatigue, a spatial visual fatigue, and a standard visual fatigue of the3D image are determined according to depth-of-field values of one ormore pixels, and a comprehensive visual fatigue is finally determinedbased thereon.

Firstly, how to determine the depth-of-field value of each pixel in the3D image or video will be described in detail in conjunction withFIG. 1. FIG. 1 is an exemplary diagram illustrating how to calculate adepth-of-field value according to a parallax according to an embodimentof the present disclosure.

As shown in FIG. 1, a 3D image is usually obtained by photographing thesame object (for example, an object P in FIG. 1) through two camerasrespectively to obtain a left eye image and a right eye imagerespectively. Without loss of generality, it may be assumed that axes ofthe two cameras are parallel and focal lengths of the two cameras areknown to be f and a distance between the two cameras (a distance betweentwo parallel axes of the cameras) is known to be b. In this case, if aparallax is d (as shown in FIG. 1, d=x₁−x₂), the following formula maybe obtained according to similar triangles, to give a depth of field zof the object P:

$z = \frac{bf}{d}$

However, this is only a simplified formula for the situation shown inFIG. 1. In a case that there are multiple cameras (multiple views),different focal lengths of the cameras, and/or non-parallel axes of thecameras, for example, a more complex formula may be used to obtain thedepth of field z according to the parallax d. In addition, in someembodiments, data of the 3D image or video per se may also carrymetadata related to the depth of field for each pixel. However, aspecific manner of deriving the depth-of-field value according to theparallax does not affect a manner of determining a visual fatigue of the3D image, and therefore will not be described in detail herein.

Next, a method for determining a visual fatigue of a 3D image or 3Dvideo will be described in detail with reference to FIGS. 2 to 5.

FIG. 2 is a flowchart illustrating an exemplary method 200 fordetermining a visual fatigue of a 3D image or 3D video according to anembodiment of the present disclosure. As shown in FIG. 2, in an optionalstep S210, the 3D image or at least one frame of 3D image in the 3Dvideo may be divided into one or more partitions. For example, in anexample shown in FIG. 3, the 3D image is divided into 9 partitions,which are numbered as partitions 1 to 9 sequentially from the upper leftto the lower right. It should be noted, however, that the embodiments ofthe present disclosure are not limited thereto and the 3D image or atleast one frame of 3D image in the 3D video may actually also be dividedinto more or fewer partitions, and shapes of the partitions are notnecessarily square. For example, the 3D image may be divided into 2partitions, 3 partitions . . . and so on. In addition, the partitionsmay also be set to a rectangle, a circle, a triangle, or any othershape. In addition, for example, the partition 5 of FIG. 3 in the 3Dimage may also be separately treated as one area and other partitionsare treated as the other area, so as to highlight an object at thecenter of a picture.

The purpose of the partitioning is to assign various partitionsdifferent weight values and obtain a comprehensive visual fatigueaccording to these weight values and the visual fatigue calculated foreach partition. The reason is that when a 3D picture is observed byhuman eyes, it is generally impossible to observe all the objects in thepicture at the same time, and crystalline lens of the human eyes areusually adjusted only for an object to be observed. Therefore, thevisual fatigue of the 3D image or video can be more accuratelydetermined by assigning a higher weight value to a partition where theobject to be observed is located while adjusting weights of otherpartitions. For example, in most of 3D movies, an object to whichattention is needed to be paid by audiences is generally in thepartition 5, objects in the partitions 2, 4, 6 and 8 are relatively lessimportant, and objects in the partitions 1, 3, 7 and 9 are basically notimportant. Therefore, weights may be assigned accordingly. For example,a higher weight is assigned to the partition 5, moderate weights areassigned to the partitions 2, 4, 6, and 8, and lower weights areassigned to the partitions 1, 3, 7 and 9. However, the presentdisclosure is not limited thereto, and weights which are the same orpartly the same may be assigned to various partitions as needed.

It should be noted, however, that this step S210 is not necessary and,in fact, the 3D image or video may not be partitioned.

Next, subsequent operations may be performed for at least one of thevarious partitions. For partitions for which the subsequent operationsare not performed, visual fatigues thereof may be considered as null andthe visual fatigues thereof may be ignored or a default visual fatiguemay be used.

In step S220, depth-of-field values of the 3D image or at least oneframe of 3D image in the 3D video may be determined. For example,depth-of-field values of all or a part of the pixels in the 3D image or3D video (or each partition) may be determined using the scheme shown inFIG. 1 or in other manners. In addition, the step S220 may be performedbefore step S210 or in parallel with step S210, and is not necessarilyperformed after the partitioning of S210.

Next, calculations related to various visual fatigues will be performed,wherein the various visual fatigues comprise, but are not limited to, aspatial visual fatigue (S230), a temporal visual fatigue (S240), and/ora standard visual fatigue (S250). In step S260, determination may bemade according to any one, two, or all of the three visual fatigues. Inaddition, the three steps may be performed in parallel, sequentially,out of order, or in any other manner, which will be described in detailbelow one by one.

How to calculate the spatial visual fatigue of the 3D image/video instep S230 will be described in detail with reference to FIG. 4. FIG. 4is a diagram illustrating exemplary pixels used for calculating thespatial visual fatigue according to an embodiment of the presentdisclosure.

Specifically, FIG. 4 illustrates a part of pixels in a 3D image whichare centered at P_(x,y). When a spatial visual fatigue of a pixelP_(x,y) is calculated, a pixel P_(x+1,y) adjacent to P_(x,y) may beacquired and a difference P_(x,y)−P_(x+1,y) between depth-of-fieldvalues of the two pixels is calculated. In some other embodiments, apixel (for example, P_(x−1,y), P_(x,y+1), or P_(x,y−1) etc.) adjacent toP_(x,y) in a different direction may also be acquired and a difference,for example, P_(x,y)−P_(x−1,y), between depth-of-field values of the twopixels may be calculated. In addition, a pixel, for example, one ofP_(x,y−2), P_(x,y+2), P_(x−2,y) and P_(x+2,y), which is spaced adjacentto P_(x,y) in various directions may also be acquired and a difference,for example, P_(x,y)−P_(x−2,y) etc., between depth-of-field values ofthe two pixels may be calculated. In addition, a plurality of pixels,for example, P_(x,y−2) and P_(x,y−1), in a range which are adjacent toP_(x,y) may also be acquired and an average value

$\frac{\left( {P_{x,y} - P_{x,{y - 2}}} \right) + \left( {P_{x,y} - P_{x,{y - 1}}} \right)}{2}$of differences between depth-of-field values of the two pixels and thedepth-of-field value of P_(x,y) is calculated. In addition, a pluralityof pixels, for example, P_(x,y−1), P_(x,y+1), P_(x−1,y), P_(x+1,y),P_(x+1,y−1), P_(x+1,y+1), P_(x−1,y−1) and P_(x−1,y+1), in a range whichare adjacent to P_(x,y) in different directions may also be acquired andan average value

$\frac{\left( {P_{x,y} - P_{x,{y - 1}}} \right) + \ldots + \left( {P_{x,y} - P_{{x - 1},{y + 1}}} \right)}{8}$of differences between depth-of-field values of the eight pixels and thedepth-of-field value of P_(x,y) is calculated. More generally, a certainset of pixels including P_(x,y) in the same frame may be acquired and anaverage value (which may be referred herein to as a spatialdepth-of-field difference of the pixel P_(x,y)) of differences betweendepth-of-field values of pixels in the set other than P_(x,y) and thedepth-of-field of P_(x,y) may be calculated. For ease of thedescription, a difference between the depth-of-field values of thepixels P_(x,y) and P_(x+1,y) is calculated as the spatial depth-of-fielddifference by taking calculations of the pixels P_(x,y) and P_(x+1,y) asan example. In addition, although the average value is used above, theaverage value may not be calculated actually, and the depth-of-fielddifferences of these pixels may be compared and/or calculated asdescribed below as long as the same calculation method is used for eachpixel.

After calculating the spatial depth-of-field differences of variouspixels, a standard deviation (or a mean square error) of the spatialdepth-of-field differences in the 3D image/video/partition may becalculated as follows:

$S_{1} = \sqrt{\frac{\sum\limits_{i = 1}^{n}\;\left( {P_{i} - P_{{ave}\; 1}} \right)^{2}}{n}}$

as the spatial visual fatigue of the 3D image/video/partition, whereP_(i) is a spatial depth-of-field difference calculated for an i^(th)pixel, P_(ave1) is P an average value of the spatial depth-of-fielddifferences, n is a number of pixels in the 3D image/video/partition,and S₁ is the spatial visual fatigue.

The purpose of the spatial visual fatigue is to present a visual fatiguecaused by different depth of fields of adjacent pixels observed by anobserver in the same picture. For example, visual fatigue is likely tooccur when the observer's point of view is switched back and forthbetween a distant object and a nearby object. Thus, the spatial visualfatigue reflects this feature in the 3D image or video. In general, theless the fatigue, the less the likeliness for the audience to fatigueduring observation.

Next, how to calculate a temporal visual fatigue of the 3D video in stepS240 will be described in detail with reference to FIG. 5. FIG. 5 is adiagram illustrating exemplary pixels used for calculating the temporaryvisual fatigue according to an embodiment of the present disclosure.

FIG. 5 illustrates two frames of 3D image in the 3D video, which are 3Dimages at times t₁ and t₂, respectively, in which each frame of 3D imageis similar to the 3D image shown in FIG. 4. In an embodiment, the 3Dimages at the times t₁ and t₂ may be two frames of 3D image which aretemporarily consecutive. In another embodiment, the 3D images at thetimes t₁ and t₂ may be two frames of 3D image which are temporarilyspaced adjacent, for example, may be spaced by one or more frames. Inaddition, in the embodiment shown in FIG. 5, the time t₁ may be lessthan the time t₂, that is, a user may firstly observe a t₁ frame andthen a t₂ frame. Of course, the time t₁ may also be greater than thetime t₂, that is, the user may firstly observe the t₂ frame, and thenthe t₁ frame.

When the temporal visual fatigue of, for example, the pixel P_(x,y) atthe t₁ frame is calculated, a pixel P_(x,y) at the t₂ frame which isadjacent to P_(x,y) at the t₁ frame may be acquired and a differencebetween depth-of-field values of the two pixels may be calculated. Insome other embodiments, a pixel (for example, P_(x,y) of a t₀ frameetc.) which is adjacent to P_(x,y) at the t₁ frame in a different timedirection (forward/backward direction) may also be acquired and adifference between depth-of-field values of the two pixels may becalculated. In addition, pixels which are temporarily adjacent toP_(x,y) at the t₁ frame in both of the two directions may be acquired.More generally, a set of pixels at different frames including P_(x,y) atthe t₁ frame may be acquired and an average value (which may be referredherein to as a temporary depth-of-field difference of the pixel P_(x,y))of differences between depth-of-field values of the pixels in the setother than P_(x,y) and the depth-of-field value of P_(x,y) may becalculated. For example, the set may comprise, but is not limited to,P_(x,y) at the t₀ frame, P_(x,y) at the t₂ frame, P_(x,y) at the t₃frame, P_(x−1,y) at the t₂ frame, and/or P_(x,y+1) at the t₃ frame etc.For ease of the description, a difference between depth-of-field valuesof P_(x,y) at the t₁ frame and P_(x,y) at the t₂ frame is calculated asthe temporary depth-of-field difference of P_(x,y) by takingcalculations of P_(x,y) at the t₁ frame and P_(x,y) at the t₂ frame asan example. In addition, although the average value is used above, theaverage value may not be calculated actually, and the depth-of-fielddifferences of the pixels may be compared and/or calculated as describedbelow as long as the same calculation method is used for each pixel.

After calculating the temporary depth-of-field differences of variouspixels, a standard deviation of the temporary depth-of-field differencesin the 3D image/video/partition may be calculated as follows:

$S_{2} = \sqrt{\frac{\sum\limits_{j = 1}^{n}\;\left( {P_{j} - P_{{ave}\; 2}} \right)^{2}}{n}}$

as the temporary visual fatigue of the 3D image/video/partition, whereP_(j) is a temporary depth-of-field difference calculated for a j^(th)pixel, P_(ave2) is P an average value of the temporary depth-of-fielddifferences, n is a number of pixels in the 3D image/video/partition,and S₂ is the temporary visual fatigue.

The purpose of the temporal visual fatigue is to present a visualfatigue caused by different time-dependent depth of fields of a pixel atthe same location observed by an observer in two or more pictures whichare temporally adjacent or spaced adjacent. For example, the visualfatigue is likely to occur when the observer observes that a certainobject is switched back and forth between a distant location and anearby location on the same pixel, and the temporal visual fatiguereflects this feature in the 3D video.

In addition, in a more common example, both of the spatial visualfatigue and the temporal visual fatigue are reflected at the same timewhen an object, for example, moves horizontally in a picture andexhibits a change in a depth of field (for example, when the objectmoves to a distant location across the picture). In this case, both ofthe two visual fatigue measures may be used to comprehensively considerthe visual fatigue of the 3D video.

In addition, in consideration that a certain visual fatigue may alsooccur even when the user observes the same still picture (even if allthe objects in the picture are at the same depth of field), it is alsopossible to introduce a concept of a standard visual fatigue. That is,in addition to the spatial visual fatigue and the temporal visualfatigue, a standard error (or a root mean square error) S₃ ofdifferences between depth-of-field values of various pixels and acertain default depth-of-field value or an optimum observationdepth-of-field value may be calculated in step S250. Thus, a defaultfatigue of the 3D image/video may be characterized by the standardvisual fatigue.

After various visual fatigues have been calculated in steps S230, S240and/or S250, the visual fatigues may be comprehensively considered instep S260 to determine a comprehensive visual fatigue. A more intuitiveway is to calculate an average value of the visual fatigues. Forexample, a final visual fatigue may be determined using a formula

$S = \frac{S_{1} + S_{2}}{2}$after calculating the temporary visual fatigue and the spatial visualfatigue. As another example, the final visual fatigue may be determinedusing the formula

$S = \frac{S_{1} + S_{2} + S_{3}}{3}$after calculating the standard visual fatigue, the temporal visualfatigue and the spatial visual fatigue. In addition, a weightingapproach may be used to determine the final visual fatigue to reflectdifferent characteristics of the 3D image/video. For example, if thereare a large number of motion scenes in a certain 3D movie, a weight ofthe temporary visual fatigue may be correspondingly increased. Asanother example, if there are both a lot of distant objects to beobserved and a lot of nearby objects to be observed in a certain 3Dmovie, a weight of the spatial visual fatigue may be correspondinglyincreased.

Next, if the 3D image/video is divided into a plurality of partitions inthe optional step S210, an overall visual fatigue may be determinedaccording to at least one partition for which visual fatigues aredetermined in an optional step S270. For example, the overall visualfatigue may be obtained by using different weights for differentpartitions as described above.

Thus, the method 200 for determining a visual fatigue of a 3D image or3D video has been described in connection with FIGS. 2 to 5. With thismethod 200, a visual fatigue of a certain 3D image/video may berelatively objectively determined, to provide the user with a widelyapplicable standard, and the scheme is easy to implement and thecalculation is simple. The embodiments of the present disclosure reflectthe fatigue value of the 3D image or video by calculating depth-of-fielddifferences at different locations and depth-of-field differences at thesame location at different times, thereby solving the problem that thevisual fatigue cannot be objectively detected.

FIG. 6 is a block diagram illustrating an exemplary device 600 fordetermining a visual fatigue of a 3D image or video according to anembodiment of the present disclosure. As shown in FIG. 6, the device 600may comprise a depth-of-field value determination unit 610 and a visualfatigue determination unit 620.

The depth-of-field value determination unit 610 may be used to determinedepth-of-field values of at least a part of pixels of the 3D image or atleast one frame of 3D image in the 3D video. The depth-of-field valuedetermination unit 610 may be a Central Processing Unit (CPU), a DigitalSignal Processor (DSP), a microprocessor, a microcontroller etc. of thedevice 600, and may cooperate with a communication portion (for example,a wireless transceiver, an Ethernet card, an xDSL modem etc.) and/or astorage portion (for example, a Random Access Memory (RAM), an SD cardetc.) of the device 600 to acquire all or a part of data of the 3D imageor video to be processed, and determine depth-of-field values of atleast a part of pixels of the 3D image or video.

The visual fatigue determination unit 620 may be used to determine thevisual fatigue of the 3D image or the 3D video according to thedepth-of-field values. The visual fatigue determination unit 620 mayalso be a CPU, a DSP, a microprocessor, a microcontroller etc. of thedevice 600, and may acquire the depth-of-field values determined by thedepth-of-field value determination unit 610, and determine the visualfatigue of the 3D image or the 3D video according to the depth-of-fieldvalues.

In addition, the device 600 may further comprise other functional unitsnot shown in FIG. 6, such as a bus, a memory, a power supply, anantenna, a communication portion, and a storage portion. However, theseunits do not influence the understanding of the principle of the presentapplication, and therefore the detailed description thereof is omittedhere.

FIG. 7 is a block diagram illustrating an exemplary hardware arrangement700 of the device 600 shown in FIG. 6 according to an embodiment of thepresent disclosure. The hardware arrangement 700 may comprise aprocessor 706 (for example, a DSP, a CPU, etc.). The processor 706 maybe a single processing unit or a plurality of processing units forperforming different actions of the flow described herein. Thearrangement 700 may also comprise an input unit 702 for receivingsignals from other entities, and an output unit 704 for providingsignals to other entities. The input unit 702 and the output unit 704may be arranged as a single entity or separate entities.

In addition, the arrangement 700 may comprise at least one(non-transitory) readable storage medium 708 in a form of non-volatileor volatile memory, such as an Electrically Erasable ProgrammableRead-Only Memory (EEPROM), a flash memory, and/or a hard disk driver.The readable storage medium 708 comprises a computer program 710 whichincludes codes/computer readable instructions that, when executed by theprocessor 706 in the arrangement 700, enable the hardware arrangement700 and/or the device 600 including the hardware arrangement 700 toperform, for example, flows described above in connection with FIG. 2and any variations thereof.

The computer program 710 may be configured with computer program codeshaving, for example, architecture of computer program modules 710A-710B.Therefore, in an exemplary embodiment when the hardware arrangement 700is used in the device 600, the codes in the computer program of thearrangement 700 comprise a module 710A for determining depth-of-fieldvalues of at least a part of pixels of the 3D image or at least oneframe of 3D image in the 3D video. The codes in the computer programalso comprise a module 710B for determining the visual fatigue of the 3Dimage or the 3D video according to the depth-of-field values.

The computer program modules may substantially perform the variousactions in the flow shown in FIG. 2 to simulate the device 600. In otherwords, when different computer program modules are executed in theprocessor 706, they may correspond to the above different units in thedevice 600.

Although the code means in the embodiments disclosed above inconjunction with FIG. 7 are implemented as computer program modulesthat, when executed in the processor 706, cause the hardware arrangement700 to perform the actions described above in connection with FIG. 2, inalternative embodiments, at least one of the code means may beimplemented at least in part as a hardware circuit.

The processor may be a single CPU, but may also comprise two or moreprocessing units. For example, the processor may comprise a generalpurpose microprocessor, an instruction set processor, and/or a relatedchipset and/or a dedicated microprocessor (for example, an ApplicationSpecific Integrated Circuit (ASIC)). The processor may also comprise anon-board memory for caching purposes. The computer program may becarried by a computer program product connected to the processor. Thecomputer program product may comprise a computer-readable medium havingstored thereon a computer program. For example, the computer programproduct may be a flash memory, a RAM, a Read Only Memory (ROM), and anEEPROM, and the computer program module may, in an alternativeembodiment, be distributed to different computer program products in aform of memory within the UE.

The present disclosure has thus far been described in connection withsome embodiments. It is to be understood that various other changes,substitutions and additions can be made by those skilled in the artwithout departing from the spirit and scope of the present disclosure.Accordingly, the scope of the present disclosure is not limited to thespecific embodiments described above, but should be defined by theappended claims.

In addition, functions described herein as being implemented by onlyhardware, only software and/or firmware can also be implemented by meansof dedicated hardware, a combination of general purpose hardware andsoftware, etc. For example, functions described as being implemented bydedicated hardware (for example, a Field Programmable Gate Array (FPGA),an ASIC, etc.) can be implemented by general purpose hardware (forexample, a CPU, a DSP) in combination with software, and vice versa.

The invention claimed is:
 1. A method for determining a visual fatigueof a three dimensional (3D) image or a 3D video, the method comprising:determining depth-of-field values of at least a part of pixels of the 3Dimage or at least one frame of 3D image in the 3D video; and determiningthe visual fatigue of the 3D image or the 3D video according to thedepth-of-field values:, wherein the step of determining the visualfatigue of the 3D image or the 3D video according to the depth-of-fieldvalues comprises: determining spatial depth-of-field differences betweenspatial adjacent pixels in the at least a part of the pixels accordingto the depth-of-field values; determining a spatial visual fatigue ofthe 3D image or the 3D video according to the spatial depth-of-fielddifferences; and determining the visual fatigue at least partlyaccording to the spatial visual fatigue.
 2. The method according toclaim 1, wherein determining depth-of-field values of at least a part ofpixels of the 3D image or at least one frame of 3D image in the 3D videocomprises: determining a parallax of each of the at least a part of thepixels; and determining a depth-of-field value of a corresponding pixelaccording to the parallax.
 3. The method according to claim 1, whereinthe spatial adjacent pixels comprise one or more adjacent or spacedadjacent pixels in a spatial direction.
 4. The method according to claim1, wherein the step of determining a spatial visual fatigue of the 3Dimage or the 3D video according to the spatial depth-of-fielddifferences comprises: determining a standard deviation of all thespatial depth-of-field differences as the spatial visual fatigue of the3D image or the 3D video.
 5. The method according to claim 1, whereinthe step of determining the visual fatigue of the 3D video according tothe depth-of-field values comprises: calculating temporal depth-of-fielddifferences between corresponding pixels in two frames of 3D image inthe 3D video; determining a temporal visual fatigue of the 3D videoaccording to the temporal depth-of-field differences; and determiningthe visual fatigue at least partly according to the temporal visualfatigue.
 6. The method according to claim 5, wherein the two frames of3D image are two frames of 3D image which are adjacent or spacedadjacent to each other in a forward time direction and/or in a backwardtime direction.
 7. The method according to claim 5, wherein the step ofdetermining a temporal visual fatigue of the 3D video according to thetemporal depth-of-field differences comprises: determining a standarddeviation of all the temporal depth-of-field differences as the temporalvisual fatigue of the 3D video.
 8. The method according to claim 1,wherein the step of determining the visual fatigue of the 3D image orthe 3D video according to the depth-of-field values comprises:calculating, for the at least a part of the pixels, a standard errorbetween the depth-of-field values of the at least a part of the pixelsand predetermined depth-of-field values, as a standard visual fatigue;and determining the visual fatigue at least partly according to thestandard visual fatigue.
 9. The method according to claim 1, wherein thestep of determining the visual fatigue of the 3D image or the 3D videoaccording to the depth-of-field values comprises: determining the visualfatigue according to any two or more of a spatial visual fatigue, atemporal visual fatigue, and a standard visual fatigue.
 10. The methodaccording to claim 1, wherein before determining depth-of-field valuesof at least a part of pixels of the 3D image or at least one frame of 3Dimage in the 3D video, the method further comprises dividing the 3Dimage or at least one frame of 3D image in the 3D video into multiplepartitions, wherein the step of determining depth-of-field values of atleast a part of pixels of the 3D image or at least one frame of 3D imagein the 3D video and the step of determining the visual fatigue of the 3Dimage or the 3D video according to the depth-of-field values areperformed for at least one of the multiple partitions, to determinevisual fatigues of corresponding partitions respectively; anddetermining the visual fatigue of the 3D image or at least one frame of3D image in the 3D video according to the visual fatigues of the atleast one partition.
 11. The method according to claim 10, wherein thestep of determining the visual fatigue of the 3D image or at least oneframe of 3D image in the 3D video according to the visual fatigues ofthe at least one partition comprises: determining a weight of acorresponding partition according to a size and/or location of each ofthe at least one partition; and determining the visual fatigue of the 3Dimage or at least one frame of 3D image in the 3D video according tovisual fatigues of respective partitions and corresponding weights ofthe partitions.
 12. A device for determining a visual fatigue of a threedimensional (3D) image or a 3D video, comprising: a processor, a memoryhaving instructions stored thereon, which, when executed by theprocessor, cause the processor to: determine depth-of-field values of atleast a part of pixels of the 3D image or at least one frame of 3D imagein the 3D video; and determine the visual fatigue of the 3D image or the3D video according to the depth-of-field values; wherein theinstructions, when executed by the processor, further cause theprocessor to: determine spatial depth-of-field differences betweenspatial adjacent pixels in the at least a part of the pixels accordingto the depth-of-field values; determine a spatial visual fatigue of the3D image or the 3D video according to the spatial depth-of-fielddifferences; and determine the visual fatigue at least partly accordingto the spatial visual fatigue.
 13. The device according to claim 12,wherein the instructions, when executed by the processor, further causethe processor to: determine a parallax of each of the at least a part ofthe pixels; and determine a depth-of-field value of a correspondingpixel according to the parallax.
 14. The device according to claim 12,wherein the instructions, when executed by the processor, further causethe processor to: calculate temporal depth-of-field differences betweencorresponding pixels in two frames of 3D image in the 3D video;determine a temporal visual fatigue of the 3D video according to thetemporal depth-of-field differences; and determine the visual fatigue atleast partly according to the temporal visual fatigue.
 15. The deviceaccording to claim 12, wherein the instructions, when executed by theprocessor, further cause the processor to: calculate, for the at least apart of the pixels, a standard error between the depth-of-field valuesof the at least a part of the pixels and predetermined depth-of-fieldvalues, as a standard visual fatigue; and determine the visual fatigueat least partly according to the standard visual fatigue.
 16. The deviceaccording to claim 12, wherein the instructions, when executed by theprocessor, further cause the processor to: determine the visual fatigueaccording to the spatial visual fatigue and one or more of a temporalvisual fatigue and a standard visual fatigue.
 17. The device accordingto claim 12, wherein the instructions, when executed by the processor,further cause the processor to: determine depth-of-field values of atleast a part of pixels for at least one of the multiple partitions, anddetermine visual fatigues of corresponding partitions according to thedepth-of-field values; and determine the visual fatigue of the 3D imageor at least one frame of 3D image in the 3D video according to thevisual fatigues of the at least one partition.
 18. A non-transitorycomputer readable storage medium for storing a computer program, which,when executed by a processor, causes the processor to perform the methodaccording to claim 1.